Mathematical and Computational Linguistics Project N.5 a Kanerva Network Model of Syntactic Parameters

Sparse Distributed Memory (or Kanerva Networks) is a technique first introduced as a model of memory in neuroscience, [2] (see also §13 of [1] for a quick summary). In a large Boolean space F2 one chooses a uniform random sample of hard locations. The number 2k of hard locations is small compared to the size 2N of the ambient space. At each of these hard locations a datum (a Boolean string of length N) is stored. Compute the median Hamming distance between hard locations. An access sphere of a point in the Boolean space F2 is a Hamming sphere of radius slightly larger than the median distance of hard locations (see some estimates in §6 of [2]). The a given location ξ in F2 , the datum D(ξ) assigned to that location is distributively stored by writing it at all the hard locations within the access sphere of ξ. When the operation is performed for a set of locations ξ, each hard location stores a datum whose i-th entry is determined by the majority rule of the corresponding entries D(ξ)i for all the stored data D(ξ). Similarly one reads at the location ξ a new datum D′(ξ) whose i-th entry D(ξ)i is given by the majority rule on the i-th entries of all the data stored at all the hard locations accessible from ξ. If the datum D(ξ) stored at ξ is ξ itself as a Boolean string of length N , this method works at reconstructing the correct datum D′(ξ) = D(ξ) = ξ because the intersections between access spheres are small and so copies of the datum where some entries are affected are in minority with respect to those copies that are faithful. When a datum is corrupted by noise, iterated readings converge to the original restored datum, provided the distance from the original is not too large. For details, see [2] and the summary in §13 of [1]. Thus, this can be viewed as a method to detect which part of a datum consisting of a binary string is sufficient to reconstruct the rest of it.